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Th ird Meeting of the Panel for Evaluati on 
of E x perimental Literacy Projects 
(Teheran/Esfahan, 8-17 September 1970) 



FINAL REPORT 



1 . The Third Meeting of the Panel for the Evaluation of Experimental 
Literacy Projects was inaugurated in the National Iranian Oil Company 
Auditors jtc, Teheran, on 8 September 1970. Subsequent sessions were held in 
the Sun and Pled Lion Society Building, Esfahan, from 9 to 1? September, 

The Panel, established in 196? for a period of 5 years to advise the 
Director-General on all problems in the evaluation of the experimental 
literacy projects, held its first and second meetings In Unesco House, 

Paris, from 11 to 15 December 196? and 2 to 6 December 1968. Panel members 
also participated in a special meeting in Untsco House, Paris, from 1 to 5 
December 1969, to consider the problems in the Experimental World Literacy 
Programme and to make recommendations for overcoming them. 

2. The meeting was attended by all members of the Fanel except 
Dr. S. K. Mitra, unable to participate due to personal reasons; 
representatives of UNDP, FAO and WHO; two consultants in cost-benefit 
analysis; the National Directors of the Iran, Madagascar, Ethiopia and 
Mali experimental literacy projects; the Chief Technical Advisers of 
the projects in Iran, Sudan and Tanzania; the national evaluation 
specialists of Iran and Madagascar; the international evaluation specialists 
from Algeria, Ecuador, Ethiopia, India, Iran, Madagascar, Mali and Sudan; 

an associate expert from CREFAL and a representative of the International 
Institute for Adult Literacy Methods, Teheran. A list of participants 
is attached (Appendix 1) . 

3. In her message to the Meeting Her Imperial Highness Princess Ashraf 
pointed out that the Government of Iran attached high importance to the 
experimental and the large scale adult literacy programme and that in view 
of the valuable experience gained it had been decided to extend the Iran 
experimental programme to the whole of the Ostan of Esfahan end the 
Shahrestan of Dezful. 

1^. His Excellency Dr. M. Rahrema, Minister of Science and Higher 
Education, referred to literacy as beiiig the symbol of better human 
conditions. In evaluating the experimental literacy projects, he stated, 
it would be essential to develop a methodology to enable an assessment of 
all the significant changes in the lives of the people. Mr. Rahnema said 
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that the Iianian experimental project had already proved a success and had 
set a challenge to the traditional concept of education. The Government 
of Iran would now introduce important changes in organizational structures 
in order to facilitate integration into and utilization of functional 
literacy experience for the large scale national literacy programme. 

5. In his address, Mr. Malcolm K. Adiseshiah, Deputy Director-General, 
enumerated the substantial progress made in many aspects of the Experimental 
World Literacy Progranme since the December 19&9 Literacy Review Meeting. 

Emphasizing however that certain basic problems were still unresolved, 
Mr. Adiseshiah requested guidance from the Panel in several areas. 

He referred particularly to the complex problems of organization and 
administration of tho experimental projects, and their links with national 
literacy programmes . It wa f important, not only to integrace the 
experimental projects noi-e thoroughly into national structures, but to 
ensure that arrangements were developed for continuation of functional 
literacy activities when the present projects ended., It was also 
essential to expand experimentation within the projects, and to develop 
adequate criteria for such experimentation in the programme as a whole. 

In the area of evaluation, it was necessary to finalize the basic minimum 
indicators to bo used In each project for international comparability, 
as well as to determine what action might bo taken in cost-benefit 
analysis. Finally, in the light of the important professional 
developments now being made in many projects, the stage had been reached 
when the diffusion of professional information must bo greatly increased, 
and means for achieving this should be considered. 

The full text of Mr. AdiseshiaMs address i3 given under Appendix II. 

6. The Panel decided imanimouoly that Dr. C. E. Beeby, Prof. Oscar Vera 
and Mr. D. Acquali should continue as Chairman, Vice-Chairman and rtapporteur 
respectively for the Third Meeting (For the Agenda please see Appendix III). 

?• . Three groups wore formed to discuss the subjects of ■experimentation 1 , 
■irtficators and international comparability 1 , ana r cost-benefit analysis 1 * 

Hr. Vora, Ifr. Lutynski and Mr. Aoquah we 'e appointed by the three groups 
as their chairmen; Mr. P# Vicillo, Mr. Bazany and fir. Sarrmak were appointed 
secretaries. Members of each group are listed in Appendix IV. 

8* Tho Panel adopted the reports prepared by the three wo: king groups 
(sec Appendix V, VI and VII). 
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Field visits in the Iran Project 



9. Representatives of the Experimental Literacy Projects in Iran 
presented the Panel with brief accounts of the project situation in 
administration arid organization, in methodology, women's programme, 
training of personnel, and evaluation. Visits were al o arranged to 
enable the Panel to gain a first hand field knowledge of th : project. 

Details cf field visits are given in Appendix VIII. 

10. *he Panel was favourably impressed by the high standard of 
professional work in the project. At the level both of conceptualization 
and of operations, systematic efforts were being made to develop all 
aspects of the pi o gramme. 

11 . Of particular interest was the methodology for curriculum and 
instructional materials. Based on observation and inquiry, the project 
draws up priority lists of problems which impede production to participant 
and group levels. The instructional objectives are then conceptualized, 
problem areas identified, appropriate sequences prepared and programmes 
developed. The literacy learning begins with a global comprehension of 
phrases related to participant experience. The processes of anal} sis and 
synthesis go side by side and inclusion of known words, exercises, drill 
and revision are used to reinforce learning. 

12. The Panel was advised that according to the experience of the Iranian 
Project the traditional method of instruction in mathematics interfereswi th 
adults logic in tackling problems, with the result that it reduces their 
computational skills. The experience shows the need for research in the 
global approach which adults apply in solving ir. a thematic a - * problems. 

13* The Panel was pleased to find that the entire professional team 
participated jn the planning and the organization of the training programme 
for instructors. The Panel also noted the close involvement of tho 
evaluation workers in ell aspects of programme development, and the well 
organized statistical data reporting system of the project, which wu& 
effectively used for feed-back and for modification of tho programme as 
required. 

Progress of World Experimental Literacy Programme 

111. Representing the Director-General, the Director of the Literacy 
Division presented to the Panel an account of the progress made in the World 
Experimental Literacy Programme with particular reference to the 
developments since the December 1969 Literacy Revievr Meeting. National 
Directors, Chief Technical Advisers and Evaluation Specialists from 
Algeria, Ecuador, Ethiopia, India, Iran, Madagascar, Mali, Sudan and 
Tanzania also made brief statements on the present situation i n their 
experimental projects. 
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15. The presentations brought out the considerable progress — 
qualitative as well as quantitative — made in warty of the projects since 
December 1969. In all projects where it was considered necessary, the 
revision of the Ple.ns of Operation is under way. The number of adults 
enrolled in literacy classes within the experimental programme now totals 
over 62,000 and this number will be considerably augmented in the coming 
year. A lrrge scale expansion stage has been reached in Iran, Mali and 
Tanzania. The basic problem of methodology, namely, the integration of 
literacy with the technical, vocational and agricultural component, 

is being resolved. Large amounts of differentiated educational materials 
are now being produced in various projects. Training activities have 
increased; regional workshops have recently been held in Addis Ababa 
and in Tunisia for national and international experts : id national 
seminars have been arranged with Secretariat asei^tance in Guinea and 
Madagascar. The dis seminal ion .<>’ information — both professional and of 
a general nature — has been expanded and to some extent systematized. 

16. In spite of these and other achievements serious difficulties 
remain. These include: 

a) Problems of organization and administration including the 
relationships between the projects and the large scale 
national programmes, ibis problem, manifesting itself in 
various forms, is probably the most difficult facing many 
projects. 

b) Evaluation delays, specially in measurement of programme 
Impact. To date the major emphasis in evaluation has been on 
providing data for programme support; for measurement of 
socio-economic impact minimum indicators have yet to be 
finalized. As for cot-benefit studies it has still not been 
determined to what extent and in what way such studies could 
be undertaken in the projects. 

c) Lack of organized policy and guidance for expe Alimentation 3nd 
research. 

Revision of Flan s of Operation 

17. The Panel notes with satisfaction that the projects have taken its 
advice to review their plans of operation where they had been found to bo 
unrealistic. The Panel at an earlier meeting made clear the principles 

it thought should be adopted in th9se reviews. It has not had time at this 
session to consider each revision In detail, although, during the present 
review members saw certain difficulties, in some of them. However, the 
revisions in general appear to bo in the direction the Panel has proposed. 
It will be helpful if tho staffs of projects could analyse more closely 
the reasons why some targets proved to be unrealistic; this world help 
now projects to avoid the same errors. It was suggested, for example, by 
speakers that some plans had not taken suffice ent account of the probable 
delays in providing staff, buildings and materials. The Panol repeated 
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its statement in earlier meetings that if projects are to be able to aiapt 
tG local needs there should bo more flexibility in financing. The Panel 
appreciates the action taken in this direction and hopes that further 
flexibility would be made. 

18. Many of the revisions obviously resulted from the fact that the 
preparatory period and the recruitment c i staff took longer than expected 
and here it seems reasonable to the Panel that, if finances permit* 
adjustments to the final date of the programme should be allowed. In other 
cases, it became clear in practice that the original assessment of the 
situation in the area was at fault. The Panel can accept the principle of 
adjustment in such cases, the actual changes to be the restlt of 
negotiations between the parties concerned. The Panel recognizes that 
formal revisions of plans of operation, requiring the signature of the 
contracting parties (UNESCO, UNDP and the Member State) take in practice 
considerable time. 

19* The Panel believes that, in all cases where changes in a program© 
arc contemplated, there should be full discussion with the professional 
officers in the project, both local and international and was glad of the 
assurance that it is the usual practice in such cases for a senior officer 
from Headquarters to be sent out for discussions. 

20. A peculiar difficulty arises when there is a difference uf opinion 
between the experts and a government on the target of nev literates to be 
aimed at within the lifetime of the international project. Some members 
felt* strongly that the adoption of targets that are too ambitious will 
interfere with the highly important duty each project has to experiment 
with new methods and materials for the teaching of functional literacy. 

The Panel repeats its previous rccomnendation that tho target adopted 
should be high enough to show the country's capacity to organize a 
significant programme on a national or regional scale, but selective enough 
to permit of genuine experimentation with methods and materials and with 
their evaluation and the evaluation of tho fj.nal results. Where quantitative 
targets are being revised the Panel feels the need to insist that the 
experimental projects are not aimed primarily at a sizeable contribution 

to the immediate eradication of illiteracy, but at assisting countries 
to find new and better ways to solve this problem in order to accelerate 
their development. 

The re l ations between the experimental projects and largo sc ale national 
programmes 

21, The reports from the projects made it clear, as + hey have in earlier 
sessions of the Panel, that there is a very great difference between the 
degree of cooperaticn projects have b 'n able to establish with national 
literacy programmes in their respect! *o countries. In one or two countries 
where the Director of the Functional Literacy Project is also the Director 
of the National Literacy Programme, there is little difficulty in securing 
the degree of coordination between the two that will enable the experiments 

functional literacy to go on having their effects even after the existing 
ojects supported by international assistance have finished, but in others 
* * kind of cooperaticn is still lacking. 



6 . 



22 . Members agreed that some kind of institutionalized adoption of 
functional literacy ideas, practices, and materials is essential in every 
country before international aid ceases, if the country is itself to have 
continuing benefit from the groat amount of work now being done. Ihe 
solution of this problem, however, will vary from country to country, 
because in any country the kind of continuing organization must conform 
to the peculiar veds, conditions, and government policies found there. 
Neither the nature of the functional literacy programme nor the 
organization set up to get continuing benefit from it can be imposed on a 
country from the outside. Each project nrast take pains to ensure that 

it leaves national agencies with concepts, materials, and methods of 
a kind that they are willing and able to use. 

23. In general it is expected that functional literacy projects will 
find their place in the overall national efforts to improve and expand 
education through the coordination of the regular school system for 
children and youth, of adult education programmes oriented to increase 
productivity and to stimulate social ard community development and of 
cultural diffusion activities. 



21j, It became clear as the debate proceeded that members of the meeting 
differed among themselves in the amount of stress they would lay on uhe 
different components of a functional literacy programme, and that these 
same differences were shown in the programmes in various countries. 

Everyone agreed that a literacy programme can claim to be functional, in 
Unesco's sons© of the term, only if it is related to the social and 
economic development of a country, but this still allows for differences 
in the weight given to each aspect of a programme and in. the types of 
material produced for it. Some programmes lay their main stress on 
specific vocational training with both the content and the methods of 
courses geared as closely as possible to a group of workers with similar 
jobs. The degree of specificity of the course will vary with the nature 
of the job, but even the most highly work-oriented projects will have 
algo more generalized courses, tuch as women's programmes centring on 
health, child care, and the like. Conversely, projects in other countries 
will have some specific work-oriented courses, but a higher percentage of 
courses of a more general nature, with more stress on social and cultural 
development • 



O 

ERIC 



25. From the beginning of the Panel's existence, there have been these 
two components in the concept of functional literacy, the one centred on 
vocational training and the provision of manpower, the other stressing 
more strongly the part literacy can play in festering broad social and 
cultural development. Both components occur in every project; only the ‘ 
emphasis varios. It is only when one recognizes the difference of 
emphasis between these two components that one can solve the problem of 
what national organizations are the natural heirs to the skills, the 
materials and the methods of each functional literacy project. It would 
be unrealistic 10 expect a national literacy programme, created for a 
broad attack on ignorance, to make much use of specialized vocational 
courses devised for highly selected groups of workers. The organizations 
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that should be encouraged to carry on that work are more likely to be the 
Kinis tries of Agriculture sud Labour, and enterprises or groups of 
enterprises actually engaged in training and employing workers who need 
skills beyond the level possible with illiterates, A national literacy 
campaign might learn much from the methods used in these courses, but 
it would not be greatly interested in their content, 

26. On the other hand, national literacy campaigns under a Ministry of 
Education should be deeply interested in both the methods and the content 
of more generalized courses dealing with such matters of common interest 
as health, child care, civic and national responsibilities, and the 
understanding of the country's culture and nationhood. One member went 
further and said that functional literacy embodies in itself a new conception 
of education in general, an education less isolated in the classroom and 
more closely related than ever before to the complex social and cultural 
goals that a country must strive for if its development is to bo rounded 

and real. So the methods and materials devised in the projects should 
influence school education as well as production oriented training and 
adult education in the broadest sense. The natural heir to this component 
in functional literacy projects might well bo the National Literacy 
Programme, though other bodies such as the Ministry of Health should also 
be interested in the findings. 

27. Members were unanimous that, whatever the body or bodies that 
might be expected to take over the lessons learnt from each section of 
each project, the work of involving them in the project must begin now. 

If it does not take place till the projects are nearly completed, it will 
be too late. 

28. Various suggestions were made as to methods of forming the 
necessary links with the appropriate national bodies: 



a) They could be told that if after the present project ceases, 
they feel the need for further technical assistance to help 
them in the continuing adaptation of the project's methods 
and materials to their needs and go on devising new materials, 
Unesco will regard very sympathetically requests for 
consultant services* 

b) If, as suggested in an earlier section of this report, 
universities can be encourricd to become involved in the 
projects, and if project officers can spend some time 
explaining to university staffs and senior students what tho 
projects are doing, there will bo a reserve of skill and 
understanding in the universities that can be put at the 
disposal of whatever national organizations take over 
responsibility for the work now being done. 



c) Similar contacts should be made as soon as possible with the 
relevant government departments. 
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d) It will also be extremely helpful to gain the understanding 

and cooperation, as soon as. possible, of teachers 1 organizations 
and of other professional bodies likely to be interested in 
the continuance of functional literacy methods • 

e) Autonony is Important to a project but it should be realized 
that it is not an end in itself but only a means of ensuring 
that the project has freedom /ta experiment . When this is once 
assured, the claim for autonomy should not be overstrassed. 

f) Training in functional literacy methods and in the preparation 
of materials should be offered to every organization likely to 
be involved in the continuance of the functional literacy work, 
and they should be invited to try out materials already produced 
and to comment on them. 

g) The general public, as well as leaders of industry and trade 
unions, should be made aware of what functional literacy pr' jects 
are doing and what they have to offer. By the time the 
international team withdraws, public opinion should be strongly 
in favour cf continuing the new methods rather than apathetic 
towards them . 

h) lines co should make it clear to its Member States that the final 
respo sibility for continuing the work of the projects will lie 
with tne governments concerned, although Unesco will give all 
the continuing technical assistance that it can. 

Experimentation with diverse methods and preparation of instructional 
materials and audio-visual aids ^ 

29. Because experimentation was to be referred to a special working 
group, the Panel decided to discuss in full session only questions of broad 
principles concerning it. One member raised the central problem of how 
specific experiments could be if their results were to be applicable on 
a world scale. This led to the debate, referred to in the last sub-section, 
on the proper balance between the vocationally oriented components ard the 
general educational components in a functional literacy programme. It was 
obvious that there was a real difference of opinion on the emphasis that 
should be given to each component, but it was not pressed to the point 
where it would prevent any project from experimenting with methods and 
materials in a way that seemed best to fit local conditions. 

There was general agreement that there should be much more 
experimentation with audio-visual media in the projects, the meeting's 
attention bo ng given mainly to the use of radio as the most immediately 
available medium. Members were agreed that there was no evidence as yet 
that audio-visual media can replace the teacher, but that they could be 
of preat help to poorly qualified and inexperienced teachers who need 
support. They could, for example, be used to enable "monitors' 1 to be used 
I functional literacy classes where fully trained teachers are not 

Available, and they could be Invaluable in predominantly vocational classes 
taught by foremen or experienced craftsmen without previous toachiig 
kiiiiiiiUflir.Ti-iiitti experience. 
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Here again the diversification o. ‘ courses raises a problem. 

It is obviously much easier to use radio in classes of a general nature, 
such as are offered frequently to women, than in very specific vocational 
classes. But experiments in the use of radio should be attempted 
whenever the conditions permit. It should be particularly valuable in 
the training of instructors and supervisors, and should be used also 
to keep them in touch with new ideas, methods and materials produced in 
the course of the work of the whole project. Needless to say, with the 
prevalence of tr'*nbistor radios in even poor countries, radio provides 
an excellent medium for explaining to the people as a whole the purposes 
and achievements of a functional literacy project. 

In the absence of television, radio is the prime method of 
influencing people's attitudes towards modernization. 

One member said that a radio transmitter is available to his 
project, and explained briefly the plans they have for using it to help 
new literates maintain their literacy. It will be used, in conjunction 
with follow-up material and rural libraries, to stimulate and guide 
group meetings to discuss matters of common interest. 

When for timing or ether reasons radio broadcasts sessions do not 
coincide with the requirements of functional literacy groups the use of 
the cassette tape recorders should be considered. 

Exchange of information 



30 » Diffusion of professional information 

The Panel believes that the time has now arrived to seek a wider 
dissemination of the pi-ofessionai information that is emerging from the 
projects. While appreciating the efforts already made by Unesco by way of 
newsletters, workshops, distribution of documentation and other 
publications, it considers that even more active methods are necessary. 

Particular attention should bo given to the universities. Some 
members expressed tl:o opinion that universities in many countries have 
given far too little attention to educational research and all were in 
agreement that the best way to make universities aware of what the 
functional literacy projects are doing is to i nvolve them in the work, 
to seek their active cooperation in research, to get them to feel a sense 
of responsibility for certain aspects of the programme. Fran every 
project there are now emerging problems that demand intensive research 
for their solution. Unesco itself is not a research organization and 
the staff of the projects in the limited time available will be mainly 
concerned with the kind of action research that is essential to guide 
the development of a project and to evaluate its results. There will 
still remain more fundamental research projects that only a university 
or research instituticn can undertake. 
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The Panel recommends that the Unesco Secretariat through its 
University Division, if need be with the help of an academic consultant, 
draw up a list of research probla s of this type that would be of value 
to the projects and of intellectual interest to university research workers, 
and that the Secretariat be asked to make those widely known in the 
universities of the world and in relevant research institutions, as a part 
of a general drive t c encour<ige res ^arch on education, with particular 
stress on adult education in developxng countries. 

The Panel feels strongly that, in this way, universities would not 
only contribute to the urgent practical problems around them, but would 
open up for themselves a rich new field of research and intellectual analysis* 
The meeting was assured by some members, including one from Iran, that many 
universities would be willing to cooperate if the position were put to them 
in clear ar.d practical terms. In seme cases, but not in all, there would 
be a need for assistance towards the cost of the research and Unesco should 
try to find funds fur this purpose. 

Other bodies through which to disseminate professional information 
about the projects include organisations such as: Unesco National 

Commissions, the International Institute for Adult Literacy Methods in 
Teheran, literacy International in India, Government Ministries and 
Departments with an interest in vocational education and functional 
literacy, CREFAL, ASFEC, and Unesco Regie nal Offices. 

It was pointed out, however, that although the universities and 
other organizations should be involved xn the work of the projects as soon 
as possible, this would not help the Secretariat with the massive job of 
collecting, classifying and storing the vast number of papers flowing into 
Paris from the projects, and with selecting, translating and editing those 
which merit distribution. This is beyond the capacity of the present 
Headquarters staff. The Panel sympathizes with the central Secretariats 
wish to make better uso of this material, but feels itself unable to 
recommend, on the information before it, how much of this material should 
be distributed or what size* of staff would be necessary to handle the work. 

In additicn to the recommendations made in this regard by the working group 
on Experimentation 1 , whose report was adopted, the Panel recommends that, 
on an experimental basis, the International Institute for Adult Literacy 
Methods in Teheran should appoint one cr two consultants for a period of 
up to a year to start this work in order to see what would be involved in 
establishing a world functional literacy clearing house. In order not to 
svamp the Teheran Institute with papers, each project should bo selective 
in the material it sends there j it is understood that the Secretariat 
will be prepared to give a lea/I in this selection. 

31 • Diffusion of general information 

The Panel i? of the opinion that this presents fewer problems than 
the diffusion of professional papers, and confines itself to saying that 
publicity for tha work of the projects should not only bo carried on through 
the general news media, but should also be aimed more specifically at civic 
authorities and village leaders where classes in functional literacy are 
O ; planned or conducted. In this connection, the Panel notes with 
sf act ion the negotiations now under way for Pro ject Support 
ini cation in several projects. 

10 
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Training of personnel 

32. In a field as novel and changirg as functional literacy, it is not 
enough to think only of the pre-service training; it must be a continuous 
process for people at all levels* Headquarters in Paris has a continuing 
part to play in the training of senior personnel* Training must be 
thought of at three levels: 

1) The training_of international .staff and of_ senior naticnal_staf f_. 
Few men and women coming freshly into a project have had experience in ~ 
functional literacy work, however great their expertise in their special 
fields* It is recognized by the Secretariat that the initial briefing in 
Paris is inadequate for the purpose. Some members thought that it would be 
better if there were more cooperation between the various sections charged 
with rre-service training, and that a full week should be spent with the 
Evaluation Section, since this is an unusually new and difficult area* 

The Representative of the Director-General assured the meeting that Unesco 
is most anxious to institutionalize the training of international staff and 
senior counterparts, but explained some cf the difficulties of applying 
this in practice* More efforts will be made in this direction; in the 
meantime short intensive workshops at the regional level will be continued# 
It was stressed that such workshops need to be arranged for specialized 
staff as well as for project leaders* In general, this is attempted. 

2) The trai ning_a t the country level of_sup£ryisors_and_trainers 1 . 

3) The train! ng_cf instructors^ 



The Panel considered the training of national personnel as 
extremely important but had no time to discuss it in any detail# However, 
it did have explained to it the system adopted in Iran and had an 
opportunity to visit training classes# It was most impressed with what it 
heard and saw, and would regard the Iranian training scheme as a 
reasonable model that could be adapted to meet local circumstances. 

Va luation and Research 

33, The Panel notes that evaluation work to date has concentrated 

more on programme support than on measurement of results. A bias in this 
direction was perhaps inevitable in the circumstances. A list of minimum 
indicators, to be applied in all experimental projects for attempting 
international comparability, has now been developed in a working group# 

In addition to the recommendations of the working group on indicators, 
the Panel invites the Secretariat to take immediately an active role in 
coordinating work in this directicn at both the technical and 
administrative levels. 
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3h* The Panel, having Insufficient time to study the demands of 
evaluation in terms of professional and financial resources available in 
each country, suggests that (Jnesco ask each project to make an inventory 
of available resources and decide on evaluation studies which would be 
feasible. It would be advisable to limit evaluation studies to well-defined 
areas and units and to achieve comprehensiveness and depth in these areas 
rather than to do superficial work in extensive areas. It is the 
responsibility of the National Directors and the Chief Technical Advisers 
to see that as far as possible all the members of the project team 
participate in evaluation. 

35* . Special emphasis must be given to the training of national 
specialists in evaluation. Sufficient counterpart staff in evaluation 
should be available to enable members to be released for trsining abroad 
for suitable periods. It would also help to arrange training and 
oriertation in evaluation for oil the project staff and as far as possible 
exports from Headquarters should be asked to cooperate in the organization 
of these courses. 

General comments and suggestions 

36. The WHO Representati\e informed the Panel that has agency was deeply 
interested in functional literacy as it could be used for health education. 

In view of the decentralization in the WHO, requests for assistance must be 
mde to the regional offices by the ministries of health of the Member 
States concerned. Indeed, in several cases, competent health officers are 
available in the national ministries of health and they could render valuable 
services for the functional literacy projects. 

37. ihe presentations made by the Secretariat and the project personnel, 
the field visits in Iran and above all the frank discussions in the Meeting 
brought out clearly the progress that has been made in the Experimental 
World Literacy Programme, particularly since the Literacy Review Meeting 
held in Paris in December 1969# The Panel expresses in this connection 

its deep sense of satisfaction. 

38. The Panel places on record its appreciation and gratitude to the 
Government and the people of Iran for their gracious hospitality. It was 
a great pleasure to meet in a country where the project of functional 
literacy has a unique support of His Imperial Majesty the Shahinshah and 
the Government. 

39* Although the present document is a report of the Evaluation Panel 
it represents in fact the consensus reached in the lieeting as a whole. 
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6. TRAINING RESEARCH AND DOCUMENTATION INSTITUTIONS/ 

INSTITUTIONS POUR LA FOHhATION, LA RECHERCHE OU LA DOCIMENT/.TIO 
Mr. Y. Ooiyens (CREFAL) 

Mr. C&stelli-Gattirara, International Institute for 
Adult Literacy Methods, Iran 



7. OTHER PARTICIPANTS / AUTRES PARTICIPANTS 

Mr. I* Friss, Cost-Benefit Analysis Consultant, 
Acade/rde des Sciences, Budapest, Hungary 

Mr. M. Oatey, Cost-Benefit Analysis Consultant, 
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Mrs R. Sohrab, Department of Guidance, Ministry of 
Education, Iran 

Mrs Nuri, Regional Director of the Esfahan Project 
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UNDp/PHtJD ifr. B. Parsons 



FAO/' Mr. A. Welle 



WHO /CI1S Dr. A. Moarefi 



UNESCO Mr. J. C. Cairns, Director, EDA/Lit 
Mr. H.P. Saksena, EDA/ Lit 
Mr, P. Vieille, Consultant in Evaluation 
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ADDRESS BY MALCOLM S. ADISESHIAH 



Ladies and Gentlemen, 

On this day five year3 ago this great city — as renowned for its 
culture as for its commitment to modernization — played host to the World 
Conference of Ministers of Education and since then every year we celebrate 
8 September as International Literacy Day* As you are well aware the World 
Conference gave a new direction to the adult literacy movement* The 
confluence of the three R's with the processes of economic and social 
development ushered in a new phase of mutual benefit and pointed the way to 
a systematic attack of the interlinked facets of underdevelopment* The 
Experimental World Literacy Programme, which is now about four years old, 
has had a rather checkered career of initial enthusiasm, then a sense of 
caution, followed by careful review and systematic replanning, leading on 
to new patterns and forms. May I take this opportunity to congratulate 
and thank you for your contribution at all the stages in the brief history 
of the Programme* 

By the middle of last year it became clear that the Experimental 
World Literacy Programme was running into difficulties. The Programme was 
reviewed by the 83rd session of the Executive Board in 19 69 and by a meeting 
of specialists convened in Paris in December 1969 to consider the ways and 
means of dealing with the situation. Since you participated in this meeting 
together with other social scientists, educators and economists as well as 
workers from the projects, I need not elaborate the conclusions you arrived at, 
I need only say that the review of the Programme has acted as stimulus to our 
project staff and with help frem all of us they have taken steps to improve 
the situation* 

The number of participants in adult literacy classes has increased* 

In Iran this year 11,1*21 persons have been attending classes in the two 
sub-projects* Increased attention has been given to the preparation of 
special functional literacy materials, training of personnel and evaluation. 

In Algeria, Guinea and Madagascar new materials have been developed. The 
Tanzania project printed 26, 000 copies of special primers for cotton and 
banana and special materials for fishing and homecraft are under piint. 

Sudan organized a four week workshop for the foremen and supervisors in 
the spinning and weaving industry. Similar workshops for writers and for 
literacy instructor were held in Tanzania and in nearly all the projects* 

The evaluation units have emphasized stidies needed by the programme 
at present. Thus in Iran a study was made on the relationship between 
demogrtphic characteristics and participant motivation and in Guinea, 
reading, comprehension and arithmetic tests were prepared. 
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Efforts are being made to further integrate the experimental 
literacy projects within the large scale national programmes. A start 
has been made to treat work-oriented projects as centres for experimentation 
and evaluation; and the tested approaches, methods and materials are 
planned to be used on a large scale. In Iran for instance, in order to 
provide a viable administrative and financial structure for the project, 
the government decided that the work-oriented project would take up the 
entire adult literacy work in Ostar* in Edfahan, and Shahrestan in Dezful. 

Some promising work has been done in devising methods and 
materials to suit the requirements of groups of participants. In Iran 
18 differentiated curricula have been developed. In Ecuador, 7 separate 
sets of materials have been produced and in toll there is a separate 
programme for each crop and for each industry. A similar trend exists 
in Tanzania. The Algerian project has prepared four films for actual 
functional literacy instructions linked to agriculture. Preparatory 
steps have been taken to purchase equipment and to train instructors 
for using TV for functional literacy in Cuenca, Ecuador. Similarly 
action has been initiated for using radio broadcasts in Ethiopia. 

During this year, the Unesco Secretariat have visited all the 
projects and have worked out with the national authorities practicable 
project and sub-project areas for literacy action, realizable quantitative 
targets and realistic timetables. Some of these missions included 
representatives from other UN agencies such as the UNDP. Plans of 
Operation aro accordingly being suitably revised. 

The 8 I 4 participants at the Functional Literacy Workshops in Addis 
Ababa and Tunis (29 April to 12 Hay, and 18 May to 6 June) discussed the 
ways of solving the difficulties actually experienced in elaborating, 
implementing and evaluating a work-oriented literacy programme. The 
results of the workshops set the stage for an intensive training programme., 
The increase in the number of the missions of Headquarters specialists 
has also contributed to the training of the field personnel. 

I hope you would find time to examine the organization of the Iran 
project in relation to the National Literacy Cmpaign, the internal 
structure and more particularly the newly developed role of the evaluation 
unit as a supporting service for programme elaboration, the differentiated 
curricula, tho weekly sequences of instruction, the integration between 
literacy and vocational training, intensive literacy teaching divided 
into two stages, namely payment of instructor remuneration in relation to 
efficiency, and the studios and action-oriented research in the field 
of functional literacy. U 10 recent developments in Iran have a 
significance for the Experimental World Literacy Programme. 




.16 



Appendix II 
Page 3 



The improvements of which I have been speaking so far point to 
possibilities and prospects. The centres which made real efforts to 
strengthen the functional orientation report high enrolment and average 
attendance figures, good literacy standards and positive trends in 
economic and social results. The qualitative changes in the experimental 
projects are, we hope, influencing the large scale national literacy 
programmes; the boundary lines should grow increasingly faint. This 
I know to be a constant preoccupation of all our Member States, namely 
how can the experimental functional literacy phase be kept short, 
how can its results be effectively channelled into the national programme. 
This also should be your constant preoccupation as a panel. 

A second area meriting your special attention is the problem of 
institutionalizing of the inter-ministerial and inter-disciplinary nature 
of functional literacy programmes. The setting up of suitable 
institutional structures, rmchinery for co-ordinated team work, 
preparation of differentiated curricula and instructional materials for 
varied groups of participants, recruitment and training of personnel at 
all levels, imaginative utilization of mass media to support the learning 
process, all add up to a large Ijst of problems that I commend to you. 

A third area on which we seek your advice is experimentation in 
thii programme. The programme is experimental as are the projects. 
Experimentation is no academic exercise but is the instrument to discover 
and improve the links between literacy and development . Can you help us 
develop the criteria for experimentation and in their light lay down a 
practical plan to be worked out in the projects dining the coming years, 
This plan should cover all aspects of work-oriented literacy, such as 
methods and materials, organizational and administrative structure, 
instructor recruitment and prepare tier,, mass -media support, local 
involvement as well as evaluation. 



A fourth subject demanding your attention is indicators. In your 
first and second meetings you Wore naturally concerned with the methods 
and procedures for evaluation from the viewpoints of their scientific 
validity, feasibility and international comparability. Have suitable 
indicators been selected ? Have adequate baseline data been collected ? 
do the methods and techniques in data collection ensure validity and 
reliability ? Has a suitable structure been set up for evaluation ? 

These are seme of the questions which have been asked in the past. 

I hope that in this meeting you will examine the indicators selected and 
applied in the projects and see h'W far they are appropriate and how far 
inter-country comparability in their case is possible. 

Finally, I an glad that you will be discussing the use of cost- 
benefit analysis techniques in the evaluation of tho experimental projects* 
This again relates to a problem preoccupying governments, ministries of 
finance, planning commission and ministers tf education who face competing 
demands for their fvtds. The limits on our own resources in the United 
Nations system and the multiplicity cf ways in which they could be spent 
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point to the need for cost-benefit analysis approach* Not only must 
costs be weighted against benefits but we nu3t also compare the rates of 
return of competing ways of investing resources. There are problems in 
quantifying cost and benefit items and calculating rates of return. 

The selection of tho discount rate* the distinction between social and 
private costs and benefits and the estimation of spillover benefits must 
be taken into account in the analysis. The problematic nature of this 
analysis relates to large scale literacy programmes at national levels 
and over long tine spans. At the community and local level and for small 
scale programmes* the coat-benefit analysis approach can yield valuable 
evidence for specific questions related to experimentation and 
qualitative programme improvements. It will also provide specific answers 
to the specific questions in the minds of the governments about the cost 
of the experiment. It should also lead to seme changes in the nature 
and content of gut experiments to reduce their cost and increase their 
benefit, 

I should like to close with an expression of gratitude to the 
Shahinshah, H. I# H, Princess Ashraf and the Government of Iran for their 
generosity which has made it possible to hold this meeting here. May I 
also thank you again most warmly for accepting the invitation to attend 
this meeting and wish you success in your deliberations over the next 
ten days. 



o 
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AGEND A 



8 Septembe r (in Teheran) 

Morning a, Reading of Message from Her Imperial Highness 

Princess Ashraf 

b* Address by His Excellency Dr, M. Ruhnema, Minister 
of Science and Higher Education 

c* Address by Mr* M* Adiseshiah, Deputy Director-General 
Unesco 



9 September (in Esfahan) 

(i) "Progress of World Experimental Literacy Programme" - 
Presentation by Unesco Secretariat 

(ii) Present situation in various experimental projects - 

Presentation by National Directors/Chief Technical Advisers/ 
Evaluation Specialists 

(iii) Discussions of these presentations with specific reference to: 

a, analysis and revision of the Plans of Operation - Choice 
of sphere of action, quantitative targets and the time 
schedule of the projects; 

b* the relationship between the experimental projects and 
large scale national programmes; 

c. experimentation with diverse methods and preparation of 
instructional materials and audio-visual aids; 

a. exchange of information with and within projects; 

e* training of personnel; 

f. evaluation and **esearch. 



11 September Weekly holiday (free) 



12-13 September 



Visit to project areas 
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lb September 

Morning Continuation of discussions listed under (iii), 

1$ September 

Group work, Ihe meeting will break into three groups, 
each will discuss one ox the following subjects? 

(i) experimentation 

(ii) indicators - international comparability 

(iii) cost-benefit analysis 

16 September 

Morning Finalization of reports in groups. 

Afternoon Discussion and adoption of group reports in the 
meeting of the evaluation panel# 

17 September 

Adoption of final report* 
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MEMBERS OF 'WORKING GFDUPS 
iraiBRSS DES CROUPES DE TRAVAIL 

GROUP I (Experimentation) 

Prof- Oscar Vera 
Mr. L.K. Meleika 
Mr. Y. Vishnyakov 
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Mr. Bonanni 
Kir. H. Kaufmann 
Mrs Nuri 

Mi*. Castelli-Gattinara 



International Comparability! / 

( Indicat eura ; Comparability Im.ernationale) 



Analysis) / Analyse du rapport cout-be^yfice) 



Mr. D, Aoquah 

Mr. R. Ramanantoanison 

Mr. A, Sammak 

Mr. B.N* Singh 

Mr. P. Wollrabe 

Mr. M. T. Hedegaard 

Mr. J. Mourad 

Mr. H.P. Saksena 

Mr. N. Khatibi 

Mr. M. Oatey (Consultant) 

Mr. I. Friss (Consultant) 



Mr. Y. Ooiyens 
Mr. P. Vieille 
Miss Tafazoli 

GROUP II (Indicators - 

Mr. A t Pouhdiba 
Mr. J. Lutynsky 
Mr. Fakoney Ly 
Mr. Rakotoarivory 
Mr. Merga Gobena 
Mr. J. C. Cairns 
Mr. A. Moarefi 
Mrs R. Sohrab 
Mr. R. Couvert 
Mr. F. Clement 
Mr. K. Bazaar 
Mr. H. Mehryar 
Mr. Khosrowchahi 



GROUP I r I (Cost -Benefit 

Prof. C. E. Beeby 
Prof. R. Dumont 
Mr. S,M. Sharif 
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REPORT OF FORKING GROUP I 
(Exp ♦ station) 



Direct oral information on Experimentation and Research activities 
being carried out or planned in each one of the projects (Algeria, Ecuador, 
Ethiopia, Iran, Madagascar, Mali, Sudan, Tanzania) ’was given to the Group 
by the corresponding specialists, national directcrs or technical advisers, 
and discussed with them. 

It appeared that all projects are engaged at varying degrees in 
Experimentation and Research and that all planned to pursue and expand 
these activities. However, it was noted that, in order to execute the 
project, more attention has been given to pilot research than to the design 
of relatively controlled experimentation which, it was agreed, would greatly 
contribute to the enrichment of the projects. 

The Group noticed that the experimentation and research being carried 
out could be classified under the following headings: 

1« Participants, including characteristics of the population 
to which the project is directed; 

2, Selection and training of personnel — instructors, 
supervisors, specialists, etc,; 

3- Methods, techniques and educational aids and materials; 

h» Organization and administration; 

5, Evaluation, 



1, The need was felt to complement the mostly quantitative information 
gathered in the base line and background studies with research in 
characteristics of the participants related to receptivity to the 
programme of the different socio-cultural and socic- economic groups. 

The Group agreed that research on values and attitudes, specific problems 
of adult learning, sub-cultural factors influencing motivation and 
learning should be encouraged, preferably with the help of universities 
and research centres. 

2, The accent has been put in most projects on the selection of 
personnel; the provisional results of these on-going experiences have 
not been systematically analysed# Little has been done concerning the 
organization, contents and methods of training. 

3- Most of the research has been done in the field of tea^lk ig materials, 
I methods and techniques for literacy work. More systematic investigation 
.s needed in methods of vocational training, agricultural extension, 
lome 
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3 . Most of the research has been done in the field of tecching materials, 
methods and techniques for literacy work, More systematic investigation 
is needed in methods of vocational training, agricultural extension, 
hone economics and particularly in mass med 1 * 1 communication. 

I;. In connection with administration and organization the on-going 
research reported by some projects deals mostly with duration and calendar 
of literacy courses. The need is pressing for more systematic research 
in this field as well as in administration (centralization or dicentral- 
ization, financing aspects, relationship with other national agencies, etc.) 

5. Evaluation is of course a common feature of all projects and 
represents most of the research activities being carried out. Refinement of 
evaluation methodology and systematic data recording are urgently needed. 

The Group hopes that cost-benefit analysis will contribute to this 
enrichment of the evaluation activities. 

The Group recogi izes that experimentation and research within the 
framework of the functional literacy projects should be meaningfully related 
to the specific needs of the projects and should be adjusted to the human 
and financial resources available to them. Within these criteria for 
selection, there is room for various approaches in research, whether 
exploratory, descriptive -diagnostic, pilot or experimental, in the 
understanding that they are all indispensable to ensure the effectiveness 
of the functional literacy programmes, beth at the national and the 
international levels. 

The Secretariat, in the light of the information on research collected 
in the different projects, is invited to prepare a guiding framework for 
research and experimentation in the World literacy Programme . This would 
help in identifying weaknesses, establishing priorities and assigning 
concrete responsibilities to each project, to the Regional Centres, to the 
International Institute and, if needed, to univerr ties and other research 
centres. 

It was felt that within this frameverk e^ch project should review its 
own plan for experimentation and research involving in it both national 
and international specialists; the Regional Centres could best 
contribute by improving the training of regional personnel in scientific 
research and carrying out basic research, and the International Institute 
should function as a clearing house and take care, with the cooperation 
of the Regional Centres, of selecting, summarizing and diffusing research 
of international relevance, and of preparing a collection of readings on 
adult education research and functional literacy problems. 
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REPORT OF WORKING GROUP II 
(Indicators - International Comparability) 



1. The task of the Group on Indicators for Functional literacy Projects 
was largely facilitated by the preparatory meetings of the evaluation 
specialists during which a tentative list of minimum and recommended 
indicators was prepared* The Group centred its discussions on the basic 
concepts underlying the indicators and considered problems in regard to 
further work in the field. 

2. The Group agreed that a more unified approach towards the assessment 
of results was necessary to attempt a comparison of results obtained in 
different projects and to draw from them some conclusions regarding the 
Experimental World literacy Programme as a whole. 

3. The Group considered the indicators as measures of various aspects 
of the teaching process and of its effects on various spheres of life. 

Not all of them have the same value for purposes of international 
comparisons and for generalizations concerning the impact of functional 
literacy. It was noted that indicators have a relative (and not absolute) 
value and that they could be properly interpreted only in relation to 
their frame of reference and methods used in collection and treatment of 
data . 



I 4 * The Group approved some 11 minimum (obligatory for all projects) 
and 19 recoiTBnended indicators, taking into account the practical 
consideration of applying them for evaluation of experimental projects. 

A list of the approved indicators is attached. 

For indicators like acquisition of literacy, knowledge and skills, 
a considerable amount of work will have to be done to develop instruments 
and measurement tools keeping in view the local conditions. 

6 . It was noted that the suggested Indicators do not cover all aspects 
of the project, specially those pertaining to instructors, methods of 
instruction, training of instructors, etc. Relevant data on these aspects 
should be obtained within the framework of experimentation in the different 
projects. Furthermore, supplementary data will have to be obtained by 
undertaking depth studies and socio-anthropological surveys. The Group 
made no recommendations concerning indicators on cost benefit analysis 
since another group was working on the subject. 

7 * The Group felt stongly that special attention should be given to 
the elaboration of precise and refined measurement instruments and tools 
so that the data obtained in the different projects could bo comparable. 
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8. The Group suggests that the Panel recommend to Unesco Secretariat 
that it should 

a) take all such steps which would facilitate the elaboration of 
methods and measurement tools for the application of the 
indicators; 

b) continue to work towards the unification of evaluation 
terminology used in the projects on the basis of the work done 
in the committee of evaluation specialists, and 

c) reassess the present situation in each project in regard to 
the personnel and technical evaluation equipment and if 
neceosary provide suitable personnel and other resources to 
enable the projects to cope up with the demands of data 
collection and analysis. Adequate personnel and equipment 
constitute the sine qua non for efficiency in evaluation. 

9. The Group hopes sincerely that all projects would now be able 
to embark immediately on the application of the minimum indicators. 
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LIST OF MINIMUM AND RECOMMENDED INDI C ATORS 



I - BASIC STATISTICAL INDICATORS 



A - Quant itative asp ects 
1* Minimum indicators 



Indicator 1«1 : Ratio of inscription in programme 

Indicator refers to the number of originally registered 
participants related to the number of places available (planned 
by the project) • 

The originally registered participants (in case when the period 
of registration exceeds the date line as defined below) are thosa 
participants who register and keep their registration up at a given 
date considered as the date of the constitution of the cohort. 

This period is defined in accordance with the given local conditions 
but it should not be longer than one month since the beginning of 
the groups* activities (instructions of topics given in the 
sequences) • 

The number of available places is the number of places planned 
by the project for a specific programme and its stage. 

The indicator is calculated in each programme after the first, 
month since registration. 

Note ? For the registered or originally registered participants 

it is necessary to give their distribution according to age 
or sex (in case of co-educational groups) • 

2 • Indicator 1.2 s Rate of drop-outs 

It is the number of participants of the cohort who left the 
course in the given stage of a programme during each month related 
to the number of participants of this cohort. 

Drop-out is a person who did not attend the course for a full 
month and/or is considered as such b/ the instructor. 

Note ? In case when the registration of participants continues 

beyond the given date line (additional registration) the rate 
of drop-outs will be also calculated out of the number of 
the total of registered participants (originally and 
additionally registered) at the end of the given month. 



0 
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Indicator I >3 • Hate of attendance 



This indicator refers to the number of actually attended 
daily sessions by all participants registered in the given month 
(at its beginning) in relation to the expected number of daily 
sessions multiplied by the number of registered at the beginning 
of the month (additional registration in the given month is not 
being taken into account) • 

Indicator l.U ; Time utilization 



This is the number of actually held daily sessions in the 
current month by the project in relation to the number of planned 
daily sessions. 

2* Recorcnended indicators 



Indicator 2*1 : Rate of coverage 

This is the proportion of the number cf available places 
(planned by the project) at the beginning of a programme out of 
the population susceptible to participate in this prograjrme. 

The number of available places is the number of places planned 
for the given population by the project for a specific programme# 

The population susceptible to participate in the programme is 
the number of individuals according to criteria of participation 
as defined by the project (age, sex, occupation, literacy status, 
etc#, for example cultivators of cotton in the age between 
1$ and U5 years). 

Indicator 2.2 : Rate of participation in final tests 



This is the number of members of the cohort who participated 
in the final examinations at the end of the stage of a given 
programme out of the total cohort* 

Note t In case of additional registration the number of those who 
participated in the examinations will be also referred to 
the total of registered in the respective stage of a given 
programme# 
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B - Qaalitative aspects 
1. Mi nimum indicators 

Indicator lw$ : Literacy acquisition 

It is the percentage out of originally registered illiterates 
who at the end of a given stage or of a given programme are declared 
as literates after having successfully passed objective examinations. 
The level, modalities of passing and marking have been fixed by a 
group of experts working in the field* 

It is recommended to develop standardized tests which allow 
the assessment of the aoquired level in the following four domains: 

a) Simple arithmetic 

b) Practical calculation pertaining to the given vocation 

c) Comprehension of read matter (e.g. technical booklets) 

d) Ability to express oneself by writing 

Indicator 1*6 : Acquisition of technical or professional knowledge 

It is the percentage out of originally registered illiterates 
who at the end of a given stage or of a given programme passed 
(criteria defined by individual project) the respective achievement 
tests. 



The it" ms of 3uch an achievement test should be drawn out of 
the prepared programmes in accordance with objectives to be 
attained by the participants (minimum level of professional 
knowledge to be acquired). 

For comparison purposes it is necessary to sutcvit the precise 
translation of the test with a copy in respective language, 
statistics describing the distribution of obtained individual scores, 
these expressed as standard scores (z-score) . 

2 . Recommended indicators 

Indicator 2 ,3 * Acquisition of knowledge of socio-economic character 

The acquisition of tb~ s knowledge covered by the programme 
should be meas.ired in an . nalogous way as in the case of the 
indicator 1,6. 
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C - Degree of adoption of writing j reading and calculating 
1 , Minimum indicators 



Indicato r 1. 7 : Use of writing ability 

Average number of personally written, sent or submitted 
messages per participant. The period of reference may be different 
according to the given categories of messages. The averages are 
being calculated for a period of one month. All written messages, 
except those submitted to the instructor, are taken into iccount 
(administrative matters, personal letters, filled in forms, etc.). 

In this indicator each message is being considered as equivalent 
to another. 

2 , R ecommended indicators 

Indicator 2,^ : Use of reading ability 

According to given possibilities one of the below given types 
of data can be used: 

a) Average number of received (and read) written messages 
per participant. The period of measurement may differ 
according to different categories of messages. Ihe average 
is calculated for the period of one month* All written 
messages, except those obtained from the instructor, should 
be taken into account (personal letters, administrative 
circulars, journals, books j booklets, calendars, etc.). 

All written messages are boing considered as equivalent. 

b) Average number of written messages arriving in the given 
locality or region per inhabitant (Sara proceedings as above). 

Indicator 2,$ : Use of arithmetical ability 

The indicator is of o,he sejne nature as the above two indicators. 
Basic datt represent the household accounts, accounts of 
associations, cooperatives, auditing, weighing, partitioning and 
other calculations and measures performed by the participants. 
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II - INDICATORS PERTAINING TO THE ECONOMICAL 
GROWTH AND DEVELOPMENT 



A - Production 

1* Minimum indicators 

Indicator 1,8 : Increase introduction per capita 

Growth of volume of the production reached in activities which 
pertain to the programme 1 s content measured in relation to the 
level cf production at the beginning cf the programme (percentage) • 
The elements of this indicator can be taken at the level of 
communities or groups of communities, enterprises, industrial 
plants or workshops and/or at the level of the groups of 
participants. 

The effects of functional literacy may be identified by 
using control groups or by other appropriate methods. 

2. Recommended indicators 

Indicator 2.6 i Quality of products 

Modify cations in the repartition of the total volume of 
commercialized production according to quality grades. Due to 
the diversity of norms established in different countries it is 
impossible to suggest a common indicator. 

Indicator 2.7 r Selling price 

This indicator can be calculated if the achieved price 3n 
the sales of products does reflect the changes in the relationship 
between producers and the market. 

The achieved price refers to the same quality and quantity. 

Indicator 2. 8 : Price (per quantitative unit) of the elements 

entering the costs of production 

Due tc diversity of elements which could be taken into 
consideration it is not possible to define in this domain a common 
indicator. For each element susceptible to observation, according 
to given conditions, the relative changes (in the time) and/or the 
relative differences (between 2 units of production) in the price 
per quantitative unit such as* insecticides, chemical manure, 
number of accidents, absenteism, purchase price, cost of 
maintenance and repairs cf machines, cost of credit, etc, will be 
calculated. 

This type of indicator refers to the community level or to 
groups in the community, to establishments, industrial plants 
or workshops. 
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B - Income - laving standard 
1* Minimum ind i cators 

Indicator 1.9 : Increase in the number of durable goods and 

Improvements contributing to the standard of living 

Increases and improvements refer to the period of one year; 
they are calculated as percentage of the number of items in a list 
prepared in advance • 

Among the goods and improvements indicating technical changes 
the following ones can be included: reduction in distance/time 

(e.g. bicycles) , control and calculation of time (watch, alarm- 
clock), intensification of communications (radio), protection of 
goods (e.g. safety locks, bolts, improvement of storing conditions), 
functional modifications of the habitat (construction of roofs, 
ventilation, lighting, etc.), improvement in water supply 
(equipment and location of wells, running water), protection 
against insects and parasites (e.g. nets, insecticides, etc.). 

The objects, products and improvements are chosen among those 
which are accessible or obtainable by the respective population. 

2. Recommended indicators 

Indicator 2.9 : Increase of net global monetary income of individuals 

The increase is referred to the volume of income measured at 
the beginning (percentage) • The effects of the functional literacy 
are identified by the control group or other technique appropriate 
for the given case. This indicator may be calculated at the level 
of groups of participants, comnunities, workshops or enterprises. 

C - Production, servicing and transport equipment 
2 . Recommended indicat or 

Indicator 2.10 : Increase in equipment for production, servicing 

or transport (in programmes for s mall 
agricultural producers ) 

Rate cf increase in the total equipment for production, 
servicing and transport (average of sevoral years). To obtain this 
rate the total equipment acquired by individuals or collectives is 
used by proprietors or others or produced locally or elsewhere. 
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D - Socio-economic attitudes 



2 . Recommended indicator 

Indicator 2,11 : Importance of changes in the domain of socio- 

economic attitudes and in the individuals 
role in society 

In the framework of this criterion it is net possible to 
define an uniform indicator: each project, according to the 

specific given status of the participants, will define its proper 
components of this indicator* 



III - ATT ITUDES T WARDS EDUCATION 

1. Minimum indicator 

I ndicator 1.10 : Rate of scholarizaticn of participants' children 

Number of scholarized children referred to the number of 
children of the participants of the schooling age. This ratio is 
given for each sax every year. 

2. Recommended indicator 

Indicator 2*12 : Rate of interruption of schooling in 

participants 1 children 

Number of participants 1 children who interrupted their 
schooling in relation to the number of interruptions of schooling 
in the total population of scholarized children. This ratio is 
given for each sex every year* 



IV - PROFESSIONAL COMPETENCE 
2 . Recommended indicators 



Indicator 2.13 : Adoption rate of recommended innovations 

This is the relation of the average number of innovations 
adopted per individual to the total number of innovations 
recommended in the field covered by the respective programme 
(percentage) . 




32 



Appendix VI 
Page 10 



Indicator 2.H+ : Level of know-how in the fields covered by 

the programme 

It is the percentage of participants who successfully passed 
one or several professional examinations pertaining to the 
know-how. In case of several examinations these are being taken 
as a single examination. 

Indicator 2.1$ : Increase in desires for changes and technical 

innovations 



Relative number of expressed desires for changes and 
innovations in a certain number of domains defined at the 
beginning of the programme. 



V - RELATION TOWARDS MEANS OF MASS COMMUNICATIONS 
2 • Recommended indicators 

Indicator 2,l6 ; Ownership of radios and television sets 

Proportion of individuals among the participants who own a 
radio and/or a television set in relation to the proportion of 
individuals who own these means of comnunication among the 
non-participants# 

Indicator 2.17 : Preference for educational programmes 

Average number (per participant) of broadcasted programmes 
of technical educational character the participant listened to 
or viewed. 
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VI - HEALTH, HYGIENE AND SAFETY 



1* Minimum indicators 



Indicator 1.11 : For the programme dealing with the problems of 

health, hygiene, nutrition, child care (most of them devoted to 
women) the indicator 1*6 will be used, as the knowledge in this 
sphere is analogous to that described under the above mentioned 
indicator. 

2. Recommended indicators 



Indicator 2*18 



and hygiene j 


, and especially in that of hygiene 


and safety 1 


[in the programmes for workers; 



The acquisition will be analysed in the same way as in the 
case of the indicator 1.6, 



VII - COSTS AND COST-BENEFIT ANALYSIS 



All the projects intend to obtain data for the studies of costs 
and ccst-benefit . The majority of the projects, however, do not dispose 
of material means and/c r do not have sufficient knowledge in this 
field* It seems necessary to give the projects appropriate support 
especially in the fern of consultants, so that they can ccraply with 
this requirement * 

2. Recormended indicators 



Indicator 2.19 : Per capita cost of functional literacy 
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REPORT OF WORKING GFD UP III 
(Cost-Benefit Analysis) 



General Report 

1* Economic criteria can play a useful role in decision making, though 
they may often be of less significance compared with social, cultural and 
political factors. However economic effects can often be reported in 
quantitative terms and as a result tend to receive greater attention than 
other qualitative outcomes. Hence great care must be taken when reporting 
results to emphasize the qualitative social and cultural factors. 

2. Cost/benefits analyses applied to training programmes is a new 
technique. Even in advanced countries studies are virtually non existent. 
Therefore techniques must be developed from scratch, and there ir> little 
concrete guidance as to what might be expected from such studies. 

3. Hence caution must be taken when indicating even the feasibility of 
applying valid cost/benefits studies to functional literacy, let alone 
predicting the sort of results that might be obtained. 

h. Nevertheless, tho group has considered a framework for performing 
limited cost/benefits analyses, and believes that they may be feasible — 
under two specific conditions: 

a) They are performed in (probably small) well defined programmes 
within selected projects at the work unit level; 

b) The programmes are expected to have explicit + effects* 

(see the attached technical report for the description of the framework and 
a definition of explicit and implicit effects) . 

5. These limited coat/benefits analyses will not at this stage be 
concerned with intra programmes comparisons such as traditional versus 
functional literacy — though they may lay foundations for such studies 
in the future* 



4- Effects are explicit (as opposed to implicit) to the extent that they are 
(l) predictable; (2) directly evaluable in economic terms; 

(3) realizable within an acceptable time horizon; (h) clearly attributable 
to functional literacy training. 
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6. These studies will help in establishing the feasibility and 
techniques of cost/benefits analysis. They nay also provide: 

(a) concrete measures of direct economic benefits; 

(b) results falling short of this which, nevertheless, anyone 
familiar with the technology in the field concerned would 
take as a reasonably fair basis for producing increased output; 

(c) information on changes in attitudes and behaviour wnich any 
reasonable person would judge to lead to developsrvent » 



The group recommends that (Jnesco provides technical assistance to 
selected projects for carrying out cost/benefits analysis. 



7« It was also pointed out t % at differences must be expected to exist 
from country to country making international comparison difficult. There 
will also bo differences from sector to sector and between industry and 
agriculture which may necessitate development of different techniques. 



8. The proposed list of indicators was examined. It was found that 
only few of them related to costs and output of production could be used 
to measure the explicit economic benefits# Most of Jhe indicators will 
provide changes pertaining to implicit benefits of the programme or to its 
cost. It was however felt that by and large the few indicators which 
could be used to measure explicit benefits may bo sufficient for the 
purpose# 

9 • In addition to coat/benefits analyses, in selected projects 
a cost analysis should be initiated in a specific project with the main 
purpose of developing standardized procedures applicable to all projects. 

The principal purpose of such analysis is however to estimate the cost 
to the national government of continuing the programmes on a non -experimental 
basis. Unesco should provide technical help and the results written up 
and distributed to all projects as soon as possible. 

10. Cost/effectivenesfl analyses can bo used to compare the costs of 
different rtfthods of achieving the same objective. Such analyses would be 
very useful for example in considering different use of instructional media. 



Technical Report 

1. The limited cost/benefits analysis studies which a re to be carried 
out in somo projects will be performed at the work unit level — usually 
the agricultural unit or the industrial firm. Macro-economic studies were 
considered to be too ambitious within the scope of the projects. 
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2. A distinction is made between explicit and irplicit benefits of 
a programme. Benefits are explicit to the extent that they are 
(a) predictable; (b) directly evaluable in economic terms; (c) realizable 
within an acceptable time horizon; (d) clearly attributable to the 
functional literacy training. Explicit benofits will be only a part of 
the total economic benefits since most of the Implicit benefits will have 
an economic impact (see Reference l). 
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functional literacy training* Explicit benefits will be only a part of 
the total economic benefits since most of the implicit benefits will have 
an economic impact (see Reference 1) * 

3. The effects of a functional literacy programme will generally appear 
at a hierarchy of levels* These are: (1) the achievement of training 

skills; (2) the acquisition of favourable attitudes; (3) the adoption of 
recommended practices; (U) the subsequent physical changes; (5) the value 
of those changes. An effect which appears normally at a lover level is 
expected to be reflected on the subsequent level; for instance the adoption 
of recommended practices in agriculture is likely to induce physical change, 
e*g* increase in a crop* 

I4 • Although the inpact of lower level effects (e.g* 1, 2, or 3) should 
be reflected at the higher levels, seme outside factors might prevent 
results recorded at an lower level being reflected, or their effect on the 
higher level might only appear after a considerable period of time (Such 
outside factors would include group norms, availability of equipment and 
the weather* (See Reference 1) * 

5* Lower level effects which cannot be reflected for various reasons 

at the higher end of the hierarchy have to be listed, emphasized and 
explained* It is even possible sometimes to make, on the basis of 
technical studies, predictions concerning the physical changes and the 
va).ue they will eventually induce* 

6. Within a project, priority for cost-benefit analysis should bo given 
to programmes where explicit effects are more likely to be anticipated* 

7* A principal difficulty in a cost/benefits analysis is to isolate the 
cause-effect relationship between functional literacy and recorded changes. 
If benefits are not on a narrow enough front to be related to the prog^une, 
tho use of tho control group technique or other appropriate method is 
recciianended . 

8., As it might be noticed, benefits of the progivw.e ore measured only 
from the production angle. This does not imply that implicit economic 
benefits which might accrue to the consumer are unimportant. They have 
been however left out to simplify the analysis framework. 

9* Cost analyses, as well as cost/bonefit analyses, will bo of a great 
help for future decision making and might also contribute to the 
improvement of the implementation of the on-going programme. 

10* The following indicators were suggested: increase in production, 

decrease in cost of production and reduction of wastage* Where these 
indicators can not bo measured the following only at a lower level might 
be applicable: adoption of recommended practices, and acquisition of 

technical or prcfcssicnal knowledge. 
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11m As far as costs are concerned, various costing approaches could 
be adopted: cost of tho programme to the central budget, to the central 

and local budget^ the total monetary costs, the real cost which takes 
into account the opportunity costs, etc. Each project should estimate 
cost 3 under all different possible approaches (See Reference 2). 

A distinction is to be made between recurrent and non-recurrent 
costs. Cost entailed ty the initial development of the programme should 
be allocated to research expenditures and considered thus as non-recurrent 
This is essential in view of the fact that the significant cost is the one 
to be incurred for the continuation of the programme by the government 
after termination of international assistance. 

12 • An appropriate deflationary index should be adopted in any cost/ 
benefits analysis in order to eliminate changes in the price level. 

Reference^ 



1. For further discussion of explicit and implicit effects see 

"The Evaluation of Functional Literacy Projects, Unesco Workshop Report 
London, 3-22 August 1969, pp. 118-131 (especially 125-129). 

2. "A Cost-Benefit Analysis of Industrial Training", B. Thomas et. al., 
British Journal of Industrial Relations, July 1969. This includes a 
suggested cost classification for training programmes. 

3. "Economics of Training with Respect to tho Firm", M. Oatey, British 
Journal of Industrial Relation, March 1970. See Part II: Benefits 
and Costs, and related sections of Part I. 



APPENDIX TO 



FIELD VISITS IN IRAN PROJECT 



12 September 1970 

i) VATAN Textile Factory 

ii) Centre No 5 for Women's handicrafts 

iii) Village Hosseynabad 

13 September 1970 

i) Instructors Training Centre, Homayunshar 
ii) Steel Mill Riz 
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